Skip to content

tecnix: read staged-only dirty paths from the git index - #75

Merged
GoodForOneFare merged 1 commit into
mainfrom
gordo-fix-staged-only-index-overlay
Oct 14, 2026
Merged

GoodForOneFare merged 1 commit into
mainfrom
gordo-fix-staged-only-index-overlay

Conversation

@GoodForOneFare

@GoodForOneFare GoodForOneFare commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Motivation

Fixes dev up in World failing with tec build errors like path '«unknown»/system/tectonix/modules/dev.nix' does not exist. It happens when a sparse checkout is mid-operation: most often an uncommitted merge of main (e.g. one stopped on a conflict), but also a stopped cherry-pick, revert or rebase. Agents in secondary worktrees hit it most, because they merge main and leave the merge uncommitted.

Context

The repo accessor sends every path git status reports to the working tree. During an uncommitted merge in a sparse checkout, git stages the cleanly merged files outside the sparse set but never writes them: they are skip-worktree entries whose only copy is the index blob. Evaluation then fails on the first one it reads, and only committing the operation clears it.

Changes

  • Parse git status --porcelain=v2 -z instead of v1, keeping each entry's status and index blob. Both names of a rename are dirty; the source of a copy is not (classified by the score field, so C. and .C alike). Unmerged and untracked paths are dirty as before.
  • An ordinary, renamed or copied entry whose worktree column is ., and whose index entry is a regular file, executable or symlink, has no change of its own against the index, so its content is that blob. readFile, readLink, lstat, directory listings and the dirty fingerprint for those paths are served from the index through the GitRepo the accessor already holds.
  • Directories that exist only in the index (a new directory the merge adds outside the sparse set, or a file it replaces with a directory) lstat as directories. A directory the merge replaces with a file now fails readDir with "not a directory", as the committed tree does, instead of listing nothing. That last case was also wrong in fully materialized checkouts.
  • Staged entries fingerprint by blob id and mode rather than content, so a directory fingerprint never reads staged blobs and doesn't depend on whether files are materialized. Staged files get a new fingerprint once (a one-time cache miss).
  • Worktree-modified, unmerged, untracked and deleted entries stay on disk, as do all dirty paths in worldtree sandboxes, which have no libgit2 repository to read blobs from.
  • Edge case: a file that is skip-worktree, staged and also edited locally now evaluates as the staged content, which matches git's view of it.

@GoodForOneFare
GoodForOneFare force-pushed the gordo-fix-staged-only-index-overlay branch 6 times, most recently from fcf29ae to d10f40c Compare October 6, 2026 12:13
@GoodForOneFare
GoodForOneFare force-pushed the gordo-fix-staged-only-index-overlay branch from d10f40c to 370a95c Compare October 13, 2026 14:53
The repo accessor sends every path `git status` reports to the working
tree. During an uncommitted merge (or cherry-pick, revert, rebase stop)
in a sparse checkout, git stages the cleanly merged files outside the
sparse set but never writes them: they are skip-worktree entries whose
only copy is the index blob. Evaluation then fails on the first one it
reads with `path '«unknown»/<file>' does not exist`, and only committing
the operation clears it. In World this reaches `tec build :__devEnv` in
any sparse worktree that merges main without committing.

Parse `git status --porcelain=v2 -z` instead of v1, keeping each
entry's status and index blob. An ordinary or renamed entry whose
worktree column is `.` has no change of its own against the index, so
its content is that blob: serve readFile, readLink, lstat, directory
listing and the dirty fingerprint for those paths from the index
through the GitRepo the accessor already holds. For a materialized file
the bytes are the same as on disk, so fingerprints are unchanged.
Worktree-modified, unmerged, untracked and deleted entries stay on
disk, as do all dirty paths in worldtree sandboxes, which have no
libgit2 repository to read blobs from.

Co-authored-by: Gord Pearson <gord.pearson@shopify.com>
@GoodForOneFare
GoodForOneFare force-pushed the gordo-fix-staged-only-index-overlay branch from 370a95c to f2b531d Compare October 13, 2026 18:03

@joshheinrichs-shopify joshheinrichs-shopify left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! thanks gord 🙇

@GoodForOneFare
GoodForOneFare merged commit 1acabb3 into main Oct 14, 2026
28 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants